A fault-tolerance protocol for parallel applications with communication imbalance

ArticuloThe predicted failure rates of future supercomputers loom the groundbreaking research large machines are expected to foster. Therefore, resilient extreme-scale applications are an absolute necessity to effectively use the new generation of supercomputers. Rollback-recovery techniques have been traditionally used in HPC to provide resilience. Among those techniques, message logging provides the appealing features of saving energy, accelerating recovery, and having low performance penalty. Its increased memory consumption is, however, an important downside. This paper introduces memory-constrained message logging (MCML), a general framework for decreasing the memory footprint of message-logging protocols. In particular, we demonstrate the effectiveness of MCML in maintaining message logging feasible for applications with substantial communication imbalance. This type of applications appear in many scientific fields. We present experimental results with several parallel codes running on up to 4,096 cores. Using those results and an analytical model, we predict MCML can reduce execution time up to 25% and energy consumption up to 15%, at extreme scale

Diseño de una infraestructura de computación de alto rendimiento para objetos paralelos en un lenguaje de alto nivel

Proyecto de investigación. Código del proyecto: 540213700005La computación paralela ha alcanzado una posición predominante en la última década gracias a la abundancia de arquitecturas computacionales de múltiples núcleos. Explotar el poder computacional disponible en los sistemas modernos ofrece una enorme posibilidad de avanzar el estado del arte en la ciencia y la ingeniería. El modelo de programación de objetos paralelos ofrece muchas ventajas con respecto a otros modelos en computación paralela. Sin embargo, este modelo no ha sido explorado en el contexto de lenguajes de alto nivel. Este proyecto se enfocó en explorar las posibilidades de diseño de un sistema de computación de alto rendimiento para objetos paralelos en un lenguaje de alto nivel. Para lograr ese objetivo se hizo una recolección exhaustiva de herramientas en el lenguaje Python para computación de alto rendimiento. Esa colección demostró la oportunidad que existe al combinar los dos dominios: objetos paralelos y un lenguaje de alto nivel. Además, el proyecto creó un panorama de las posibilidades de diseño de tal combinación

Algoritmos alternos de bajo coste para la comparación de rutas metabólicas en plantas

Informe Final de Proyectos de Investigación y ExtensiónMetabolic pathways provide key information to achieve a better understanding of life and all its processes; this is useful information for the improvement of medicine, agronomy, pharmacy and other similar areas. The main analysis tool used to study these pathways is based on the idea of pathway comparison, using graph data structures. Graph comparison has been defined as a computationally complex task. We propose two algorithms with different approaches which simplify the problem of comparing pathways represented as graphs. The first algorithm consists in the transformation of a two-dimensional graph structure to a one-dimensional structure, and thus aligning the corresponding data using a reduced 1D structure. The second algorithm consists in performing a pair analysis between graphs, that is to say a relation of 2 equal nodes present in both graphs, and thus eliminating all similarities, finally, showing these differences to the user. Our results show evidence of a quick, simple and effective way to resolve the described problem. The mechanism proposed in algorithm 1 can be used as a prior evaluator to predict good comparisons in case a deeper analysis is desired. We show that the loss of information or precision does not affect much the result, which is to give the user a similarity score between the two analyzed pathways. For algorithm 2 the proposal is to offer the expert an additional point of view for his evaluation of the pathway in question. In this case, no score is provided but the listed differences

Using migratable objects to enhance fault tolerance schemes in supercomputers

Supercomputers have seen an exponential increase in their size in the last two decades. Such a high growth rate is expected to take us to exascale in the timeframe 2018-2022. But, to bring a productive exascale environment about, it is necessary to focus on several key challenges. One of those challenges is fault tolerance. Machines at extreme scale will experience frequent failures and will require the system to avoid or overcome those failures. Various techniques have recently been developed to tolerate failures. The impact of these techniques and their scalability can be substantially enhanced by a parallel programming model called migratable objects. In this paper, we demonstrate how the migratable-objects model facilitates and improves several fault tolerance approaches. Our experimental results on thousands of cores suggest fault tolerance schemes based on migratable objects have low performance overhead and high scalability. Additionally, we present a performance model that predicts a significant benefit of using migratable objects to provide fault tolerance at extreme scale

Timed consistency: unifying model of consistency protocols in distributed systems

Ordering and timeliness are two different aspects of consis- tency of shared objects in distributed systems. Timed consistency [12] is an approach that considers simultaneously these two elements according to the needs of the system. Hence, most of well known consistency proto- cols are candidates to be unified under the Timed consistency approach, just by changing some of the time or order parameters.Red de Universidades con Carreras en Informática (RedUNCI

A Study of Checkpointing in Large Scale Training of Deep Neural Networks

Deep learning (DL) applications are increasingly being deployed on HPC systems, to leverage the massive parallelism and computing power of those systems for DL model training. While significant effort has been put to facilitate distributed training by DL frameworks, fault tolerance has been largely ignored. In this work, we evaluate checkpoint-restart, a common fault tolerance technique in HPC workloads. We perform experiments with three state-of-the-art DL frameworks common in HPC Chainer, PyTorch, and TensorFlow). We evaluate the computational cost of checkpointing, file formats and file sizes, the impact of scale, and deterministic checkpointing. Our evaluation shows some critical differences in checkpoint mechanisms and exposes several bottlenecks in existing checkpointing implementations. We provide discussion points that can aid users in selecting a fault-tolerant framework to use in HPC. We also provide takeaway points that framework developers can use to facilitate better checkpointing of DL workloads in HPC

Understanding soft error sensitivity of deep learning models and frameworks through checkpoint alteration

The convergence of artificial intelligence, high-performance computing (HPC), and data science brings unique opportunities for marked advance discoveries and that leverage synergies across scientific domains. Recently, deep learning (DL) models have been successfully applied to a wide spectrum of fields, from social network analysis to climate modeling. Such advances greatly benefit from already available HPC infrastructure, mainly GPU-enabled supercomputers. However, those powerful computing systems are exposed to failures, particularly silent data corruption (SDC) in which bit-flips occur without the program crashing. Consequently, exploring the impact of SDCs in DL models is vital for maintaining progress in many scientific domains. This paper uses a distinctive methodology to inject faults into training phases of DL models. We use checkpoint file alteration to study the effect of having bit-flips in different places of a model and at different moments of the training. Our strategy is general enough to allow the analysis of any combination of DL model and framework—so long as they produce a Hierarchical Data Format 5 checkpoint file. The experimental results confirm that popular DL models are often able to absorb dozens of bit-flips with a minimal impact on accuracy convergencePeer ReviewedPostprint (author's final draft

Power, Reliability, Performance: One System to Rule Them All

En un diseño basado en el marco de programación paralelo Charm ++, un sistema de tiempo de ejecución adaptativo interactúa dinámicamente con el administrador de recursos de un centro de datos para controlar la energía mediante la programación inteligente de trabajos, la reasignación de recursos y la reconfiguración de hardware. Gestiona simultáneamente la fiabilidad al enfriar el sistema al nivel óptimo de la aplicación en ejecución y mantiene el rendimiento a través del equilibrio de carg

Mejoramiento del modelo de la estructura interna de capas y corteza del Volcán Turrialba

Instituto Tecnológico de Costa Rica. Escuela de Ingeniería en Computación. Informe final. Código : 5402-1370-0007La actividad volcánica tiene un efecto importante en las actividades humanas y la infraestructura. Las recientes erupciones volcánicas de los volcanes Poás y Turrialba han impactado económicamente a las comunidades circundantes: algunos parques nacionales y aeropuertos han debido cerrarse temporalmente; ganado, pobladores y escuelas se han tenido que reubicar. El reconocimiento de esta amenaza sirve como motivación para que las autoridades locales y la comunidad científica use infraestructura moderna de computación para mejorar nuestro entendimiento de los fenómenos vulcanológicos. Esta propuesta implica la construcción de una plataforma de computación avanzada para mejorar el modelo de la estructura interna de capas de un volcán y la ubicación de temblores volcánico-tectónicos. Toda esta información, junto con modelos teóricos, ofrecerá un mejor entendimiento de la dinámica del Volcán Turrialba

Framework para Simulación en Paralelo de Fenómenos Sismológicos y Vulcanológicos

Proyecto de investigación. Código del proyecto: 1370005Costa Rica es un país situado en el llamado Cinturón de Fuego del Pacífico, una zona altamente sísmica que comprende países en ambos extremos del Océano Pacífico. En Costa Rica, en promedio, se experimenta un sismo de magnitud 4.0 o superior diariamente. Es fundamental para el país contar con una plataforma computacional para entender mejor los fenómenos sismológicos y el efecto que pueden tener los sismos en la sociedad. Este proyecto tuvo como objetivo principal identificar las necesidades de simulación y procesamiento de datos de los observatorios sismológicos del país (OVSICORI y RSN) y construir un framework que permitiera ejecutar esos programas. El entregable principal fue una primera versión del framework para obtener sismogramas sintéticos. Se diseñó una plataforma que simula sismos computacionalmente y que a la vez asocia información geográfica para crear videos del sismo con información del entorno físico. Esta integración permite una visualización enriquecida de los fenómenos. El framework integra varias herramientas de código libre que ejecutan en arquitecturas paralelas y que tienen la capacidad de simular una amplia variedad de escenarios. Este tipo de infraestructura es esencial para el país y demuestra el potencial que existe en la colaboración científica y el uso de tecnologías de computación avanzada